# Efficient Web Applications with Erlang and Cowboy

## The Web is made for Erlang

### Loïc Hoguin - @lhoguin

#### Erlang Cowboy and Nine Nines Founder

# The Web yesterday

# The Web today

# What makes it possible?

- XmlHTTPRequest
- EventSource
- Websockets
- SPDY

# What does the Web needs exactly?

- Efficiency
- Concurrency
- Availability

# Efficiency?

- Low latency
- Low bandwidth footprint
- Low memory footprint

# Concurrency?

- High number of concurrency connections
- True concurrency: each request has the same weight
- No loss of latency due to an expensive connection
- No loss of latency due to many connections

# Availability?

According to Joe Armstrong, you need:

- Isolation
- Concurrency
- Failure detection
- Fault identification
- Live code upgrade
- Stable storage

# Trine

- Efficiency requires concurrency
- Concurrency requires availability
- No point measuring efficiency if the server is unavailable

# Erlang, Erlang, Erlang

- Erlang did this long before it became important for the web
- Erlang did this long before multicore CPUs
- Erlang gives you all this for free

# Erlang models the real world

- People don't share memory
- People evolve concurrently
- People communicate by sending messages
- People die, for a reason we can identify
- The world doesn't crash when someone dies

# Error handling and recovery

- Erlang features pattern matching
- Error handling is straightforward: "let it crash!"

          ok = file:write_file(Name, Contents)

- Application design depends on how you want them to fail
- Your system becomes failure-safe, even from your own bugs

# Erlang is easy to read and write

- Code is written sequentially
- Asynchronous code is hidden behind a module API
- The language is very small and easy to understand
- Modules are written in a top-down manner

# Erlang systems grow naturally

- It's almost the same code to go from 1 to 2 nodes
- It's the same code to go from 2 to 1000 nodes
- It's the same code to handle local or remote faults or netsplits

# What does a HTTP server look like?

- Accept connection
- Receive and parse request line
- Receive and parse header lines
- Do something
- Send a reply
- Maybe process another request

# This is what Cowboy is doing

# Then is Cowboy sequential?

- Accepting the connection in a separate process
- Many processes accept connections concurrently
- Requests are processed concurrently

# Is it really effi-- YES!

- Prove it!
- Sure.

# And then I realized...

- Testing efficiency is hard!
- Many tools exist to benchmark web servers
- Problem: they don't scale as much as Cowboy

# Trying out Siege

    % siege -b -c 100 -t 30s http://192.168.1.101:8080
    ** SIEGE 2.70
    ** Preparing 100 concurrent users for battle.
    Transactions:                 518565 hits
    Availability:                 100.00 %
    Elapsed time:                  29.76 secs
    Data transferred:               5.93 MB
    Response time:                  0.01 secs
    Transaction rate:           17424.90 trans/sec
    Throughput:                     0.20 MB/sec
    Concurrency:                   98.86
    Successful transactions:      518565
    Failed transactions:               0
    Longest transaction:            0.06
    Shortest transaction:           0.00

# Bigger Siege

    % siege -b -c 1000 -t 30s http://192.168.1.101:8080
    ** SIEGE 2.70
    ** Preparing 1000 concurrent users for battle.
    Transactions:                 106280 hits
    Availability:                 100.00 %
    Elapsed time:                  52.41 secs
    Data transferred:               1.22 MB
    Response time:                  0.37 secs
    Transaction rate:            2027.86 trans/sec
    Throughput:                     0.02 MB/sec
    Concurrency:                  747.29
    Successful transactions:      106281
    Failed transactions:               0
    Longest transaction:            6.26
    Shortest transaction:           0.00

# The thing about benchmarks

- They're never a "fact" to anyone other than yourself
- Perform your own benchmarks before taking decisions
- Always check where the bottleneck is

# The Horse project

# Sweet Poney benchmark

- Loops of "Connection: close" requests
- Start with 500 concurrent clients
- Add 500 clients every 3 seconds
- 5K clients running after just 27s

# Hardware

- Same hardware type for client and server
- Motherboard: ASUSTeK Computer INC. M5A78L-M LX
- AMD Athlon(tm) II X4 640 Processor (3000 Mhz)
- 4 GB of 1333 Mhz memory

# Hello world results

# Raspberry Pi results

# Raspberry Siege

    % siege -b -c 100 -t 30s http://192.168.1.14:8080
    ** SIEGE 2.70
    ** Preparing 100 concurrent users for battle.
    Transactions:                   7986 hits
    Availability:                 100.00 %
    Elapsed time:                  29.58 secs
    Data transferred:               0.09 MB
    Response time:                  0.37 secs
    Transaction rate:             269.98 trans/sec
    Throughput:                     0.00 MB/sec
    Concurrency:                   99.27
    Successful transactions:        7986
    Failed transactions:               0
    Longest transaction:            0.52
    Shortest transaction:           0.20

# 1KB static file

# 100KB static file

- Benchmark client breaks down
- When it doesn't it's serving about 1000 files/s
- Couldn't get a proper graph
- But I don't recommend serving files from Erlang

# Integrate Cowboy in your applications

- Experimental FCGI handler
- rake for Rails
- Direct some queries to Erlang and others to your existing system

# Hello PHP FCGI

    % siege -b -c 100 -t 30s http://192.168.1.101:8080/hello.php
    ** SIEGE 2.70
    ** Preparing 100 concurrent users for battle.
    Transactions:                  65535 hits
    Availability:                 100.00 %
    Elapsed time:                  29.94 secs
    Data transferred:               0.75 MB
    Response time:                  0.03 secs
    Transaction rate:            2188.88 trans/sec
    Throughput:                     0.03 MB/sec
    Concurrency:                   56.34
    Successful transactions:       65535
    Failed transactions:               0
    Longest transaction:            0.43
    Shortest transaction:           0.00

# Experimental FCGI code is experimental

# wsdemo benchmark

# Real world application stats

- Dating website chat, peak time
- 70k sessions online
- 35k active websocket connections

# Is it webscale?

- Tested with 1 million active websocket connections on Amazon EC2
- Takes about 20GB of memory
- That's about 20KB per connection

# Is there a catch?

- Yes
- We haven't started optimizing Cowboy

# How is this possible?

# Erlang the OS

- Erlang the VM runs a release
- Release contains applications
- Applications contain processes
- Processes are supervised
- Erlang the VM is also supervised

# Real world Erlang system design

- Each connection is its own process
- Each session is its own process
- Session data is kept inside the process
- Live events are kept inside the session process
- Cache is often done internally in your application
- Results in few database and FS calls

# Erlang system design 101

- Think about how you want your system to fail
- Protect your core services into inner layers
- Just let outer layers fail
- Results in small focused services applications
- They only do one thing and do it well

# Architecture of an Erlang web app

- Thin HTTP communication layer
- Makes calls to services in your system
- Services are live processes, not a model library
- Services can be on this or another node
- Connection processes are mostly waiting for replies
- They do not prevent other connections from running!

# Scale to the infinite and beyond

- Remove bottlenecks (core processes...)
- Distribute responsability (master-less)
- Let it crash (but handle faults where it matters)
- Reduce disk or network access
- Measure

# Erlang gives you the tools to measure

- Default installation contains:
  - 4+ profilers
  - 2+ debugging interfaces
  - 3+ trace interfaces
  - Direct peeking into running processes
  - Tons of statistics
  - SNMP
  - A lot more...

# What you don't need anymore

- MVC and friends
- Cache servers (memcache, redis...)
- Messaging servers (Rabbitmq, 0mq...)
- Overtime

# Getting started

- https://github.com/extend/cowboy
- http://learnyousomeerlang.com
- #erlang and #erlounge on irc.freenode.net
- erlang-questions mailing list
- Contact me directly! @lhoguin or essen@ninenines.eu

# Questions?